Method and System of Transitive Matching for 
Object Recognition, in Particular for Biometric 
Searches 

5 Field of Invention 

The present invention relates to the fields of object and pattern recognition and more 
particularly to a method and system of recognizing or matching an object such as the biometric 
information of an individual. 

10 Background of the Invention 

Statistical object recognition techniques are generally performed on sets of selectively 
measured data, referred to as features, to identify and distinguish objects or patterns of interest. 
Each collection of feature measurements may be considered a multi-dimensional vector, with 
each individual feature corresponding to one of the vector components. Each feature thereby 

1 5 forms an axis of a multi-dimensional feature space, and consequently any feature vector has a 
defined position in the feature space. The details of feature extraction depend on the 
application. One category of object recognition is biometric identification, where feature data 
representing the unique physical personal characteristics of an individual are used to recognize 
or verify the identity of a person. Commonly used biometric identifiers include face images and 

20 fingerprints, although other identifiers such as an individual's voice or hand geometry are also 
used. 

In most biometric recognition techniques, before recognizing or matching the biometric 
information of anjndividual, a high dimensional biometric vector space is constructed . In 
biometric recognition, the raw biometric sample data obtained fro m indivi duals, corresponding 
25 for example to a fa ce image or a fingerprint, is encodgd into a biometric Jeaturej^ect that 
"exists" in the biQ meiric3^ector.space. The similari ty or nearness o f a probe biometric vector 
in comparison to other previousl v„encQded _biometric vectors can then b e measur ed, to 
determine a match and recognize an individual. Generally a match is made with the pre- 

encoded biometric vector that is most similar or nearest to the encoded biometric vector bein g 

30 ■ ^ 

tested or p robed. 

For example, in face recognition systems, a^gaUery of facg imagesjsjirst enrolled i n the 
system and encoded for subsequent searchin g. A probe fac e is then obtained and compared with 
eac h coded face in the gallery, and the individual is recognized when a suitable match occurs. 
Because human faces and other biometric identifiers may display significant variation in 

35 

appearance due, for instance, to changes in fa cial expression, orientation, facial hair, or lighting 
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conditions, it is often difficult for face recognition systems to identify faces which exhibit 
considerable variation in appeara nce. 

One well-known technique for face recognition uses eigenvecto r codin g derived from 
Principal Components Analysis (PCA) to represent faces for recognition. In this photometric 
5 algorithm, t he eigenvectors of the covariance matrix for an initial trainijig^et of face images 
are determ ined. The eige nvectors (which represent the principal components of the training set) 
act as a set of featur es which together characterize the variation between face images. Each fage^ 
location contributes to a varying.^gree to each eigenvector, and the eigenvectors can be 
displayed as an which deviates frorn unj^orrn^grgy where certain fa cial jfe a tures 

10 differ from those of the training..set faces. In this manner, the gigenfaces map the variation^ 
between faces , and each face can be represented in terms of a linear combination of the 
eig£nfaces. Since the eigenfaces (and corresponding eigenvectors) account for a different 
amount of the variation among the face images depending on their ei g envalues, by using^ 
subse t of the eigenfaces having the largest eigenvalues, the most significant variation within 
1 5 the image set can be approximated with reduced dimensionality . The pro jectio n of an image on 
to t his lower dime nsional^geMpacg.yJel^^ encoded^ctor. This projection, which is in 
effect an overlap ima ge score, is then used for recognition by comparing the position or score j 
of that encoded ve ctor in eigenspace with the position or score of known individuals, and then ' 
determining whi ch js^ neare s t . Eigenface recognition techniques are described in more detail by 
20 Turketal., "Eigenfaces foTrecognit\on'\ Journal of Cognitive Neuroscience 3(l):71-86 (1991), i 
the contents of which are incorporated herein by reference. 

Several other methods and techniques for object, and more particularly biometric 
recognition, have been developed. For example, in addition tOMeigenfag^es, prior art recognition 
systems may be based on: dynamic link matc hing as disclosed by Lades et al., "Distortion 
25 invariant object recognition in the dynamic link architecture", IEEE Trans on Computers, 
42(3):300-3 1 1 (Mar. 1 993), probabilistic matching as disclosed by Moghaddam et al., "Beyond 
Eigenfaces: Probabilistic Matching for face recognition", FG '98, pp. 30-35 (1998); support 
* vectors as disclosed by Phillips, "Support Vector Machines applied to face recognition" 
Advances in Neural Information Processing Systems 11, eds: Jordan, Keams, and Solla (1998); 
30 and local feature analysis as discussed by Penev et al., "Local Feature Analysis: A general 
statistical theory for object representation", Network: Computation in Accural Systems 7(3), 477- 
500 (1996) and as incorporated in the Facelt® software developer kit available from Visionics 
Corporation of Jersey City, New Jersey. The contents of each of the references cited above is 
incorporated herein by reference. 

35 
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Although the accuracy of these and other recognition methods is typically good, they 
are nevertheless generally unable to rapidly and efficiently carry out the required searches for 
determining a match in a large database of objects or individuals. In many applications, this is 
problematic since object or biometric searches need to be performed in minimal time, e.g. for 
5 a real-time recognition application. There is therefore a need for a system and method of object 
recognition capable of improving the speed of existing object or biometric recognition 
algorithms. It would also be desirable if the amount of data required to store or represent 
encoded biometric or object vectors could be reduced or compressed. 



10 Summary of the Invention 

The present invention provides a method and system of transitiyit y^matchin g^u^ 
for object recognition, and in particular biometric recognition such as face or fingerprint 
recognition. The invention provides a highly efficient means for pre-encoding obj ect m atch 
information from each setjQfja w ob j ect sample data, such as a face image or a fingerprint, to 
1 5 be stored in^^^d^abase or to be checked agains£^^t abase. With the transitiv e recognition 
method and system of the present invention, the time required for the recognition system to 
perform matching is dramatically reduced compared to that of an underlying object recognition 
algorithm. In addition, the size of the encoded object vectors or templates is also significantly 
reduced compared to the size of raw object vectors output by the underlying object recognition 
algorithm. For this purpose, the transitive recognition method and system of the present 
invention provides constructs a v gctor space in to which a data sample rel ating to an object is 
encoded, for recognitjon purposes. The construction of these new vector spaces and the 
encoding of object data in to these vector spaces enables the above described advantages of the 
present invention to be provided. 

Thus, in a first aspect, the present invention provides a method of constructing a v ector 
space^ in which^^^d^a^samp^l^ relating to an ^ol^ect jnay be encoded. The method comprises 
providing a raw matching score between each of a pluralit y of basis sample elements and each 
o£a plurality of data,sampl es in a first sample database . The samples in the first sample 
database are out-of-sample with respect to t he bas is samples. The method also comprises 
constructing a sample space from the raw matcjiing scores, with the sample space being defined 
by a basis set of sample space modes. Constructing the sample space may comprise generating 
a covariance matrix for the basis elements from the raw matching scores; and determining the 
eigenvectors and eigenvalues of the covariance matrix, the eigenvectors specifying the sample 
space modes. The method may also comprise generating a rotation matrix, based on the 
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eigenvectors of the covariance matrix, the rotation matrix mapping raw matching scores into 
sample space scores. 

In one embodiment, the sample space is also truncated by eliminating a subset of the 
sample sp ace m odes, the eliminated modes generally having theJo_we st e igenvalues. This may 

5 comprise determining a sample space score between each of a plurality of data samples in a 
second sample database and each of the sample space modes, the samples in the second sample 
database being out-of-sample with respect to the basis samples and the samples in the first 
sample database. A cut-off for eliminating a subset of sample space modes based on the 
distribution of said sample space scores for each of the sample space modes is selected, the 

1 0 distribution being indicative of how well a sample space mode is able to discriminate between 
samples. 

Preferably, the method also comprises: determining a first common object sample space 
score between each of a plurality ofdata samp les in a first common object sample databas e and 
. each of the sample space modes; determining a second common object sample space score 
' ^ between each of a plurality of data samples in a second common object sample database and 
each of the sample space modes, the first and second common object sample databases each 
having a set of different samples for the same objects; and constructing a recognition space 
from the first common object sample space scores and the second common object sample space 
scores, the recognition space being defined by a basis set of recognition space modes. The 

20 

sample space scores may be normalized with respect to each sample space mode before 
constructing the recognition space. Constructing the recognition space may comprise generating 
a second covariance matrix for the sample space modes from the first common object sample 
space scores and the second common object sample space scores; and determining the 
eigenvectors and eigenvalues of the second covariance matrix, the eigenvectors of the second 

25 

covariance matrix specifying the recognition space modes. A second rotation matrix may then 
be generated, based on the eigenvectors of the second covariance matrix, the second rotation 
matrix mapping sample space scores into recognition space scores. 

Preferably, the recognition space is truncated by eliminating a subset of the recognition 
space modes, the eliminated recognition space modes generally having the highest eigenvalues 
of the second covariance matrix. Truncating the recognition space may comprise determining 
a recognition space score between each of a plurality of data samples in a third sample database 
and each of the recognition space modes. The samples in the third sample database being of 
objects not sampled in the basis samples, the samples in the first sample database, the samples 
in the first common object database, or the samples in the second common object database. A 
cut-off for eliminating a subset of recognition space modes based on the distribution of said 
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recognition space scores for each of the recognition space modes is then selected, the 
distribution being indicative of whether a recognition mode is generally unsuitable for 
recognizing different samples of the same object. 

Alternatively, the recognition space may be constructed using linear discriminant 
5 analysis or non-linear discriminant analysis. Preferably, the plurality of basis sample elements 
are selected at random. 9^ 



f^s>p^or\ In another aspect the present invention provides a method of encoding a data sam ple 
relating to an obje c^o^enable the object to recognized. The sample space, truncated sample 
space, recognition space, and/or truncated recognition space are constructed as above. A raw 
matching score between the data sample and each of the plurality of basis sample elements is 
then provided. The raw matching scores of said data sample may then be mapped into sample 
space, truncated sample space, recognition space, and/or truncated recognition space scores. 

In another aspect, the present invention provides a method of recognizing whether a 
probe data sample matches one of a pjurality of known data samples, comprising encoding the 
known data samples and encoding the probe data sample into sample space, truncated sample 
space, recognition space, or truncated recognition space. The distance between the sample space 
scores for the probe sample and each known sample in the sample space is measured, allowing 
which encoded known sample is nearest to the encoded probe sample in the sample space to 
be determined. 

In another aspect, the present invention provides, for use in object recognition, a space 
construction and encoding system comprising: a sample space construction module for 
receiving a data sample relating to an object to be encoded, a plurality of basis sample elements, 
and plurality of data samples in a first sample database. The samples in the first sample 
database are out-of-sample with respect to the basis samples. The sample space construction 
module provides a raw matching score between each basis element and each of the plurality of 
data samples in the first sample database, and constructs a sample space from the raw matching 
scores, the sample space being defined by a basis set of sample space modes. 

The system preferably also includes a recognition space construction module for 
receiving a plurality of data samples in a first common object sample database and plurality 
of data samples in a second common object sample database. The first and second common 
object sample databases each have a set of different samples for the same objects. The 
recognition space construction module determines a first common object sample space score 
between each of the plurality of data samples in the first common object sample database and 
each of the sample space modes and fiirther determines a second common object sample space 
score between each of the plurality of data samples in the second common object sample 
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database and each of the sample space modes. The recognition space construction module then 
constructs a recognition space from the first common object sample space scores and the second 
common object sample space scores, the recognition space being defined by a basis set of 
recognition space modes. The sample space scores may be normalized with respect to each 

5 sample space mode before construction the recognition space. The plurality of basis sample 
elements are preferably selected at random. 

The system also preferably includes a sample space truncation module for truncating 
the sample space by eliminating a subset of the sample space modes, and a recognition space 
truncation module truncating the recognition space by eliminating a subset of the recognition 

^0 space modes. 

The objects may be persons, the data samples may represent faces of the persons, and 
the method may be used to perform face recognition. Alternatively, the objects may be persons, 
the data samples may represent fingerprints of the persons, and the method may be used to 
perform fingerprint recognition. 



Brief Description of the Drawings 

The objects and advantages of the present invention will be better understood and more 
readily apparent when considered in conjunction with the following detailed description and 
accompanying drawings which illustrate, by way of example, preferred embodiments of the 
invention and in which: 

Fig. 1 is a block diagram of a preferred embodiment of the transitive recognition system 
of the present invention; 

Figs. 2 A-2B are diagrams illustrating how the transitive property is used in the transitive 
recognition method and system of the present invention; 

Fig. 3 is a flow chart showing a preferred method of operation of the sample space 
rotation module of the transitive recognition system of Fig. 1; 

Fig. 4 is a flow chart showing a preferred method of operation of the sample space 
truncation and measure module of the transitive recognition system of Fig. 1; 

Fig. 5 is a flow chart showing a preferred method of operation of the recognition space 
rotation module of the transitive recognition system of Fig. 1; and 

Fig. 6 is a flow chart showing a preferred method of operation of the recognition space 
truncation and measure module of the transitive recognition system of Fig. 1. 

Detailed Description of Preferred Embodiments 



15 
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Although the description which follows relates primarily to biometric recognition 
techniques such as face or fingerprint recognition, it will be appreciated that the transitive 
recognition method and system of the present invention may be generally extended to 
recognition of other types of objects or patterns. 
5 In biometric recognition, the comparison of data for a number of biometric identifiers 

does not return a Boolean (true/false) matching score when one data sample set is compared 
against another. Instead, the recognition algorithm returns an overlap or distance score, S, based 
on the notion of nearness or similarity. This is the case for face and fingerprint recognition 
systems, but is also commonplace in other types of object and pattern recognition systems. 
10 Since bia metric scores obey a transitive _pxo.p.ertv, ^ described below, g _score comp aring the 
bio mg tries of different individuals nia y_contain relevant and meaningful match information 
even though the individuals do not represent a match. 

Fig. 1 is a block diagram illustrating how a transitive recognition system 100 of the 

present invention may be used with an underlying object or biometric recognition system 1 50 

to improve the speed and efficiency of recognition matching. Conceptually, transitive 

recognition system 100 preferably includes four main sof tware modules: a jam ple space 

construct ion and scorejota tion module 11 0, a sample space truncation and measure module 

120, a recognition space construction and score rotation module 130, and a recognition space 

truncation and measure module 140. During operation, these modules use certain obj ect o r 
20 ' - " - ^ 

bio metric sampl e databases 1 70, as shown in Fig. 1 and described in detail below. System 100 

^y;^^-^ |. 

<^^0^^ of the present invention exploits the transitive_pj-opertY^e^ri_hed_abQve ta.pxavid.e_muclijaster i 

and more efficient recognition and matching than that available with the underlying recognition 

system and algorithm 1 50 alone. In effect, the underlying recognition algorithm 1 50 gener ates 

' Vaw" scores that ac t as inpu ts to transitive sys tem 100 . The underlying or raw recognition 

25 ^ " ' ' — ' 

system 150 may use any suitable reco^nitionalgQrjthm for Renerating mw matcj iingscores and 

may be based, for example, on eigenface , d ynamic lin k matc hing, probabUisti^n^^ or 

support jvecto^Jecjiniques_ as referenced abov e. For example, recognition system 150 mayj 

capture sample biometric data.fr.orn_an_individual--160-gei ierate a raw score ba sed_on that 

sampj e^da^ ^'^^^^^''^^J^^ ^N^ixA.ic^ 



30 



The transitive propert yisjllustrated conceptually in Figs. 2A and 2B. Referring to Fig. 
2A, instead of directly comparin g the biometric data of sample A with the biometric data of 
sam ple C , the data of samples A and C are independently compared with a third basis or 



reference s ampleJB . By virtue of the transitive property, if samp le A ha s a high score S(A) 
against sample B and sample C also has a high score S(C) against sample B, then there is a high 

:>5 

likelihood (i.e. there is an increased probabiHty) that sample A has a high score against sample 
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C. In the present invention, the transitive property is applied to an underlying match algorithm 
using a plurality of basis sample element s, shown as B, to B^b in Fig. 2B (and contained in 
database DBb in Fig. 1) As described in detail below, recognition may be much more rapidly 
established by comparing the scores S(A ) of sarnple A wit h each basis element B i to Bnr to the 
scores S(C) of sample C with the same basis elements , as fundamentally illustrated in Fig. 2B. 
In this manner, the score vectors S(A) and S( C)_each provi de encoded recognition inform aUon^ 
that may be use d to measure nearness in a biometric jy^c^r space^crealed^^ elements 
B 1 to BjMB. Different recognition measures for determining the similarity or nearness of the score 
vector S(A) = {S,(A), S.CA),... SMB(A)}^and the score vector S(C)= {Si(C), S.CC),... Smb(C)}'^ 
can be used. For example. 



score of sample^A^ith the I'th basis^ekment. Other measures which determine nearness based 
on, for example, Euclidean distance or Manhattan distance are also possible. Generally, the 
smaller the value o f M(A,C ), the clo ser encoded vectors S( A) and S(C) are in the biometric 
vector space create d by the basis elements Bj to B^b, and the higher the probability of a match 
or recognition between samples A and C. 

However, because the set of basis elements is a finite se t, a transitive recognition 
measure - such as M(A,C) - that determines the similarity between the scores S(A) and S(C) 
does not produce optimal results for the classification or recognitio n of biometr ics. In 
accor dance with the present invention, transitive recognition system 1 00 perforin sjjirst sam ple^ 



space rQtatio n,.and preferably also a second recognition space^rotation, on raw scores obtained 
^ TronTunderl ^dng^ecjORnilion syste m 150 to prov ide a highly compressed encoded vector that 
enables rapid matching and recognition to be performed with an improved transitive measure. 
The method of operation of transitive system 1 00 is depicted in flow chart form in Figs. 3-6 and 
is described below. 

Fig. 3 shows a preferred method of operation of sample space rotation module 1 10 of 
transitive recognition system 100 of Fig. 1. Referring to Fig. 3, transitive system 100 must 
initially select the basis elements Bj to B^jb in database set DBb- Each basis element B, 
repr esents the biometrics of a differe nt indi vidual and m a y be sele^ted ^co^ding t o different 
^ criteria. For example, the basis set DBg may be taken from representatives of the population 
having the highest overlap (or scores) with non-basis probe biometric sam ples. On the other 




where 
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hand, the basis set DBg may be taken from representatives of the population having minimal 
overlap with each other, thereby helping to ensure that the vector space created by the basis set 
covers the extreme boundaries of the biometric space well. As a further alternative, the 
elements in the basis set DBg may be sej^ected at random . In the specific context of face 
5 recognition, all three of the above basis constructions were attempted, and the randomly 
constructed basis set provided the best recggmtion during out-of-sample testin g (i.e. testing on 
biometric samples not in basis set DBr ). As a result, a randomly selected basis set is preferably 
selected in step 310, at least in the case of face recognition. 

However, since it is difficult to determine a truly random subset over a biometric 
^0 population, there is a strong possibility, particularly for small basis sizes, that certain parts of 
the biometric space may be over-represented and certain parts may be under-represented. The 
existence of over-represented regions of the biometric space leads to over-discrimination in 
favor of biometric samples in the over represented space. Conversely, there is under- 
discrimination against biometric samples in the under-represented regions of the biometric 
space. As described below, in view of the inherent limitations of a finite size^ basis DBg, 
transitive system 100 provides an improved transitive recognition measure t hat maps the "raw " 
scores S(A) and S(C) into newly constructed biometric vector spaces. 

To determine such an improved measure that can be used to denote the distance 
between two biometric samples in biometric space, a sample database.^Bg is used, and so is 
selected at step 320 in Fig. 3. The database DBg includes N S_samp.les, Aj to A^^g? each of 
which is out-of-sample with respect to the b asis set e lements in DB r (the samples A, to A^s in 
database DBg may also be referred to as test samples ). At step 330, for each basis element B, 
to Bj^B, and each representative Aj to A^^ in database DBq, the sc ore Sf A) is calculated using 
the underlying matching algorithm. A NB x NB covariance matrix COV is constructed, as 
shown in step 340, where the (I, J)'th element of the matrix COV is given by 

COV,j—^'£'" [S,(A)-M,l[SjCA)-Mj] 

NS^ ■ 

and where S"^^ denotes a sum over DBj with A = 1 to NS, and with M, (and similarly Mj) given 
by: 

' NS^ ' 

35 It should be noted that the matrix COV is computed using out^jjf-sam ple.data. i.e. the scores 
S(A), allowing the transitive property to be exploited. Furthermore, it will be appreciated that 
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the covariance matrix COV provides a convenient way of determining and assessing how the 
dispersion of the basis elements is related. However, although the covariance matrix COV is 
preferably used to generate sample space scores as described below, it will be appreciated that 
other measures of dispersion (i.e. other than variance) may also be used. 

5 If each of the raw space basis element directions I and J, where I ^ J, are completely 

de-corr elate d, then the off-diagonal elements ofCOV would-b e zero, and the I'th diagonal 
elements would equal the variance in the I'th basis direction (with the variance o~ equaling the 
square of the standard deviation o). As will be appreciated, however, in the "raw" b iometric 
vector space of DBb th e basis directions I and J will generally be correlated to varying degrees. 

1^ The COV matrix is real and symmetric (COV = COV") and can hence be diagonalized into a 
form that reveals the matrix's eigenvectors (i.e. (j), to 4)nb) corresponding eigenvalues 
A, (i.e. A| to A^b), as indicated at step 350. Specifically, a linear transformation that diagonalizes 
COV provides a matrix of eigenvectors. The diagonalization may be accomplished by a 
numerical technique such as singular value decomposition or a Householder-QR (or 

^ ^ Householder-QL) decomposition. 

Since COV is real symmetric , its eigenvalues are ^non-negative and its ^ eig envectors 
form a complete orthogonal_s et. The resulting eigenvectors (J)i denote the linearly independent 
(or principal component) directions or modes E, in a biom etric sample space S] and the 
eigenvalues X, indicate the square of the "width" of the distribution of samples in those 

20 

directions, i.e. the I'th eigenvalue represents the variance in the I'th direction. At step 360, the 
eigenvectors and values are ordered in terms of decreasing eigenvalue A, such that A,sib ^ ^nb-i 
< ... < A2 < A]. 

It should be noted that the raw scores S,(A) may be processed before being used to 
generate COV or, more generally, to construct the sample space Sj. For example, the scores 

25 

S,(A) may each hav e a mean subtracted from them to compensate for a certain condi tion (e.g. 
lighting) during the acquisition of samples. Furthermore, more than one score may be used for 
each person or object represented in DBg. 

A biometric sample space rotation matrix C is determined, at step 370, by 

30 

C = [(i)j (j)2 ... (t)NB] 

where C maps between the raw biometric space and the biometric sample space S,. The rotation 
matrix C denotes the most important directions or modes E, in biometric S, space, i.e. the 
35 directions that best separate one sample from another and hence provide a maximal 
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discrimination between biometric samples. At step 380, a scale for each direction E, in the 
biometric sample space S, is also determined and given by 

D, = Dnb < Dnb-, ^ - < D2 < D, 

5 

with D, representing the standEird deviation in the linear independent direction E,. Again as 
indicated, the sample space S, is ordered with the largest scales D, indexed before the smaller 
scales. The scales D, are used to normalize the effect of each direction E, in a sample space 
score generated at step 390 as: 

10 

where denotes a sum over the initial basis set DBq with J = 1 to NB, and SE,(A) refers to 

J ^ the sample space score of sample A with the E, direction or mode in the biometric sample space 
(i.e., rotated from the raw biometric space). It will be appreciated that due to the normalization 
with the scales D,, the entire biometric sample space S, becomes a normalized hyper-sphere. 

Small D, represent less relevant directions E, in which linear combinations of the scores 
in the basis have the least ability to discriminate between samples in the database DBs. These 

2Q low scale directions may also be thought of as "fme-tuned" directions to the database DBg, and 
therefore are not suitable for biometric discrimination. As a result, the recognition method and 
system of the present invention preferably truncates the biometric sample space S, by 
eliminating modes E, with small scales Dj, before generating a transitive sample space measure 
ME(A,C). Fig. 4 illustrates, in flow chart form, a preferred method of operation of sample space 

25 truncation and measure module 120 in transitive recognition system 100. To determine an 
appropriate^^off in^Ejjnodes, a second out-of-sample database DBs2 rnay be used (i._e. DBs2 
is out-of-sampl e with respect to _ho.th.DBR.and DBg). Database DBs2 is therefore selected at step 
410. The sample space scores SEi(A) for each sample A ii^DBsg^are calculated for each mode 
Ej of the biometric sample space S, in step 420. At step 430, the standard deviation aS2i of the 

30 distribution of the scores for each mode is then determined, at step 430, and divided by the 
corresponding D, at step 440. If the vector rotation for that mode generalizes, the ratio aS2,/D, 
should be close to 1 .0, for example from about 0.9 to 1 . 1 . However, as this ratio approaches 1 .3 
- 1 .4, the vector analysis clearly fails for the out of sample data in DBs2. It will be appreciated 
that increases in the ratio. oS2,/Di closely correspond with decreases in Dj for the ordered modes 

35 E, in biometric sample space. Thus, an appropriate cut-off may be taken for sample space 
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modes E, for which this ratio begins to exceed some value within the range 1.1 and 1.3, as 
indicated at step 450. In one preferred embodiment, a cut-off ratio of approximately 1.25 is 
selected, corresponding roughly to three standard deviations beyond the expected value of 1 .0. 
In this manner, at step 460, a number of the E, having the lowest Dj are simply truncated, based 
on the ratio cut-off above, from the biometric sample space S, thereby further increasing the 
robustness of the method and system of the present invention to out of sample biometric data 
(i.e. data out-of-sample with respect to both DBb and DBs). The number of principal directions 
or modes E, remaining after truncation is denoted as NE, and an improved measure for 
biometric discrimination in the truncated sample space S, is given, as indicated at step 470, by 

M^(^,Q=-i-5:^'^ [SE^{A)-SE^{C)f 
NE 

Again, other similar (Euclidean, Manhattan, etc.) measures of the distance between biometric 
vectors A and C in truncated sample space can alternatively be used. 

The sample space into which the rotation matrix C maps, as constructed by modules 110 
and 120 in system 1 00, is still not optimal for performing biometric or object recognition. This 
is because the sample space measure ME(A,C) does not take into account the possibility that 
the same object or individual may undergo variations in appearance that, insofar as recognition 
matching is concerned, should be disregarded. As a result, transitive recognition system 100 
provides for a further rotation of the sample space scores SE(A) into a recognition space SR,. 
The recognition space SR, further takes into account the possibility that in the sample space S,, 
there may be directions associated with changes of the same individual or object, such as a 
change of expression in a face, or a change of orientation or scale of a fingerprint. As indicated, 
these directions in sample space are or should be directions of zero measure size in recognition 
space. Stated another way, the recognition space SR, includes biometric sample space 
directions ER, that differentiate between different individuals. 

A preferred method of operation of recognition space rotation mjodule 130 is illustrated 
in Fig. 5. To construct the biometric recognition space SR,, a database pair, defined as two 
databases DBp, and DBp2 each having a different biometric sample of the same NP individuals. 



is selected at step 510. That is, the samples between databases for the same database index 



differ only by a re-sampling of the same individual, for example at different times, in different 
surroundings, and/or at different orientations. As the database entry changes in DBp, and DBp2, 
the individual sampled changes as well. The databases DBp, and DBp2 may also be referred to 
as common object databases. Recognition space SR, is built on biometric sample space Si by 



- 12- 



NY2 - 1094897.2 



10 



15 



20 



25 



using the database pair and, in effect, identifying the directions in biometric sample space that 
best differentiate between individuals. 

Referring to Fig. 5, at step 520, for each mode (i.e., in the truncated sample space 
modes) and each sample pair in the databases DBp, and DBp2, the sample space scores SEi(Al) 
and SE,(A2) are calculated, where Al and A2 are representative samples of the same individual 
(or object). It should be noted that the biometric sample score SE(A) (as opposed to the raw 
score S(A)) is used. A NE x NE covariance matrix COVR is constructed, as shown in step 530, 
where the (I, J)'th element of the matrix COVR is given by 



COVR^j=—^Y.^'^ [SE^(A1) -SE^(A2)] • [SE/A1)-SE/A2)] 

where S^** denotes a sum over the database pair DBp, and DBp2 with Al, A2 = 1 to NP. The 
matrix COVR is again real and symmetric and can be decomposed or diagonalized into a form 
that reveals the matrix's eigenvectors (|)R, (i.e., c|)R| to 4>Rne) ^^id corresponding eigenvalues 
ARi (i.e., ARi to ARj^j^), as indicated at step 540. As with the diagonalization of COV, singular 
value decomposition or Householder-QR techniques may be used to diagonalize COVR. 

The resulting eigenvectors <^K^ form a complete orthogonal set and denote the linearly 
independent directions ER, in biometric recognition space SRj. The eigenvalues AR, are non- 
negative and represent the variance or the square of the "width" of the distribution of samples 
in the directions ERj. At step 550, the eigenvectors and values may be denoted or ordered in 
terms of increasing eigenvalue XR, such that AR, < AR2 ... < A.Rne.] <ARj^e. 

At step 560, a biometric recognition space rotation matrix CR is determined, at step 

560, by 



CR=[(|)R, (i)R2 ... (1)Rne] 

where CR maps between the biometric sample space Sj and the biometric recognition space 
30 SRj. The matrix CR denotes the most important directions ER, in recognition space SRj, the 
directions that best separate one person (or object) from another, and hence provide a maximal 
discrimination between individuals. At step 570, a scale DRj for each direction ERj in the 
biometric recognition space is determined and is given by 



DR, - (XR,)'/= DR| < DR2 < ... < DRjsjE-i < DR^ 
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with DR, representing the standard deviation in the ERj linear independent direction in 
recognition space SR,. The directions and scales are ordered with the smallest scales DR| 
indexed before the larger scales. 

The input elements of the COVR matrix, SE,(A)5 are representatives in each sample 

5 space direction E, of distributions with a variance equal to 1 (normalized to 1 by dividing by 
Di). As indicated, the sample space Si corresponds to a normalized hyper-sphere of "size" 1 in 
all sample space directions Ej, If the underlying algorithm used in the raw recognition score 
system 150 is unable to perform recognition, then the diagonalized elements of COVR (i.e. its 
eigenvalues AR,) will be approximately equal to 1, since this is the variance in the biometric 

1 0 sample space Sj, and the rotation performed by matrix CR does not affect this property. On the 
other hand, if the underlying raw score algorithm is able to perform recognition, then the 
distribution in biometric sample space Si of the difference between scores of samples of the 
same person (i.e. SEi(Al) - SEi(A2)) is smaller than if the samples correspond to different 
people. Thus, in the recognition space analysis, the space is ordered with respect to the smallest 
scales DR, first, since the smaller the scale, the better the direction or mode ERj is for 
recognition. 

A rotated recognition space score is generated at step 580 as follows: 
SER^(A)^Yl^^ CR'SE/A) 

20 

where denotes a sum over the truncated sample space modes Ej with J = 1 to NE and 
SERi(A) refers to the recognition space score of sample A with the direction or mode ER, in 
the biometric recognition space. 

2^ In the alternative to constructing the recognition space SRj by calculating and 

diagonalizing a COVR matrix as described above, other methods and techniques may be used 
on the projections of sample database pairs from DBpi and DBp2 on to sample space S,. 
Furthermore, other common object databases (e.g. DBp3, DBp4, etc.) having additional distinct 
samples of the same objects or persons in DBpj and DBp2 may also be used to construct 

OQ recognition space. Some of these disciminant techniques are linear, such as Fisher's Linear 
Discriminant analysis which takes a difference between vectors in sample space Sj, and then 
applies Fisher analysis along with subsequent diagonalization techniques; while other 
discriminant techniques are non-linear, such as algorithms based on two or three layer neural 
networks. These and other suitable techniques are described by Bishop in "Neural Networks 

35 for Pattern Recognition" (Clarendon Press: Oxford, 1995), the contents of which are 
incorporated herein by reference, 
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Where the scales DR, for each direction or mode ER, in recognition space are 
approximately equal to 1, this is indicative of directions in which linear combinations of the 
scores in the recognition space basis have the least ability to perform recognition in the 
databases DBg, DBpj, and DBpj. These directions may again be considered "fine-tuned" 
5 directions to the databases DBg, DBp,, and DBp2 and should not be used for person or object 
discrimination. Recognition space truncation and measure module 1 40 in transitive system 1 00 
therefore preferably carries out a truncation of the recognition space directions ERj before 
providing a transitive recognition space measure MER(A,C). 

A preferred method of operation of module 140 is shown in Fig. 6. To determine an 
appropriate cutoff in modes, a third out-of-sample database DBgs (i.e. DBs3 is out-of-sample 
with respect to DBg, DBs, DBp,, DBp2, and DBs2) whose samples correspond to different 
individuals or objects than those in other databases, may be used. Such a database DBs3 is 
selected at step 610. The recognition space scores SER,(A) for each sample A in DBgs are 
-calculated for each mode ERj of the biometric recognition space SR, in step 620. As described 
above, since the samples in DBs3 correspond to different individuals or objects than those in 
other databases, no recognition should occur, and ideally the scales DRj should be one. 
Referring back to Fig. 6, the standard deviation oS3i of the distribution of the scores for each 
mode is then determined, at step 630, and divided by the corresponding DR, at step 640. If the 
vector rotation for that mode generalizes, the ratio oS3,/DRi should be close to 1 .0, for example 

20 

from about 0.9 to 1 . 1 . In contrast, as this ratio approaches 1.3-1 .4, the vector analysis clearly 

fails for the out of sample data in DBs2. It will be appreciated that in this case, increases in the 

ratio aS3,/DR, considerably beyond 1 .0 closely correspond with increases in DR, considerably 

beyond 1.0 for the modes ER, in biometric sample space. 

Thus, an appropriate cut-off may again be taken for recognition space modes ER, for 
25 ... . 

which this ratio begins to exceed some value within the range 1 . 1 and 1 .3, as indicated at step 

650. Again, in a preferred embodiment, a cut-off ratio of approximately 1 .25 may be selected, 

corresponding to about three standard deviations beyond the expected value of 1.0. In this 

manner, at step 660, a number of the ER, generally having the highest D, are truncated, based 

on this ratio cut-off, from the biometric sample space SR,. As a result, the robustness of the 

30 

method and system of the present invention to out-of-sample biometric data (i.e. data out-of- 
sample with respect to both DBq and DBg) is further increased. The number of principal 
directions or modes ER, remaining in the truncated recognition space SR, is denoted as NER. 
An improved measure for biometric discrimination in the truncated recognition space SR, is 
^ given, as indicated at step 670, by constructing: 
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MER{A,C)^ 



1 



NER 



where, for example. 



5 RF^^ 



DR^ 



-1 



/ 



The recognition factor RF, is used to ensure that, as the mode ERj has lessening recognition 
capabiHty, DR, approaches 1, so that the mode ER, does not contribute to the measure. The 
recognition factor RFj is one of many possibilities, and generally RFj only needs to ensure that 



contribute strongly to recognition receive more weight. The above implementation of 
recognition factor RFj provides one such simple solution. In addition, other measures of the 
nearness of points or vectors SER(A) and SER(C)) in recognition space SRj can alternatively 
be constructed. 

The recognition space measure MER(A,C) is a main result of the transitivity principles 
outlined above. The construction of the recognition space SR, (and the sample space S,) 
provides a vector space in which encoded biometric or object sample vectors SER can exist. 
The encoded vectors are generated by taking the underlying or core recognition algorithm in 
system 150 and the database DBg of basis elements to first provide the raw scores S(A) = 
{Si(A), S2(A),... Snb(A)}'^ of a probe or test sample A. The vector S(A) is then mapped or 
rotated into the sample space scores SE(A) = {SE,(A), SE2(A),... SE^eCA)}^ to provide a vector 
encoded in sample space. Preferably, the sample space encoded vector SE(A) is then further 
mapped or rotated into the recognition space scores SER(A) = {SER, (A), SER2(A),... 
SERmer(A)}^. Thus, the distance or nearness between any two recognition space encoded 
vectors SER(A) and SER(C) can be measured with a suitable measure MER(SER(A),SER(C)) 
which can equivalently be given as MER(A,C) as described above. When it is desired to match 
a probe person (or object) with one of a number of known persons (or objects), the encoded 
score of a known person that provides the lowest measure score of MER(A,C) generally 
determines a match or recognition. 



Transitivity matching in this manner provides a means for pre-encoding biometric 
match information from each set of raw biometric (or object) sample data, such as a face image 
or a fingerprint. With the transitive recognition method and system of the present invention, the 
time required for the recognition system to perform matching is dramatically reduced, i.e., the 
time required to generate a recognition space encoded vector SER(A) for a probe sample A and 
compare it, using MER(SER(A),SER(C)), to other similarly encoded known samples C is 



0 modes ERj that are unable to perform recognition do not contribute, and modes ER, that 
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dramatically reduced. In addition, the size of the encoded biometric vectors or templates 
SER(A) is also significantly reduced compared to the size of raw score vectors S(A). 

In effect, the sample space S, is a linear approximation to the non-linear recognition 
properties of the underlying core algorithm in system 1 50, however the recognition space SR, 
5 has "knowledge" of recognition beyond that provided by the initial algorithm in system 150. 
Although the transitive system 100 preferably constructs (and encodes object or biometric 
vectors in) recognition space SR,, in some cases constructing (and encoding vectors in) only 
the sample space S, may be beneficiaL For example, this may be the case where object 
recognition is being performed on objects that do not generally vary over time. Also, while 
^ 0 truncating each of the sample and recognition spaces is preferred, truncation is not required to 
obtain the benefits and advantages of the present invention. The major advantage of more 
rapidly matching objects is due largely to the pre-calculation of scores and the exploitation of 
the transitive property. 

The sample space may be populated via any reasonable score generator, and there are 
no special requirements for the underlying score generation algorithm in system 150. For 
example, most, if not all, face recognition algorithms can be used to provide raw score inputs 
to transitive system 100 of the present invention. Similarly, most imaging based fingerprint 
algorithms may also be used to provide raw score inputs to transitive system 1 00. Furthermore, 
the speed of many other biometric and object recognition algorithms may also be significantly 

20 

improved by the transitive system and method of the invention. The principal criterion is that 
the raw score algorithm in the underlying recognition system 150 must be able discriminate 
between individuals or objects, as required by the application. 

In general, the accuracy of the measure MER(A,C) of the present invention follows the 
accuracy of the underlying algorithm in system 150, as the number of elements in the basis 

25 

database DBb becomes sufficiently large. For example, in a face recognition application, the 
basis database DBg preferably includes 200 or more basis samples. As a result, there is little 
or no loss of accuracy with respect to the underlying match algorithm in system 150. 
Furthermore, in the process of constructing the recognition space SRj, recognition data is 
provided by way of the database pairs DBp, and DBp,. This additional data may serve to 
improve the accuracy of the initial core algorithm. 

In one preferred embodiment of the present invention, for a face recognition application, 
raw recognition score system 1 50 includes components of the Facelt® software developer kit 
available from Visionics Corporation of Jersey City, New Jersey. For example, system 1 50 may 
include the function CFACE::LoadFace of the Facelt® developer kit which generates face 
templates or faceprints from video images using a Local Feature Analysis algorithm. In 
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addition, system 150 may include the function CFACE::IdentifyPerson of the Facelt® 
developer kit which generates a representation, i.e., a digital faceprint, for faces and performs 
matching between different representations to generate a raw matching score. 

In a specific example of such a face recognition implementation, the following database 
5 sizes were used: DBq included 200 samples of different people; and DBg, defining biometric 
sample space, contained 1,000 samples of different people. The truncated biometric sample 
space included the first 128 modes E, (i.e., the 128 modes having the largest D,). The database 
pair DBpi and DBp2 included sample pairs of 1000 individuals, and the truncated biometric 
recognition space included the first 80 modes ERj (i.e., the 80 modes having the largest DRj). 
In this implementation and with these parameters, the time required to encode and match a 
biometric sample is roughly one second, i.e., 200 matches per second for a raw score vector 
S(A)on a 500 MHZ Pentium III computer system. 

After construction of the SER(A) vector, which in its final form is 80 dimensional, the 
SER(A) vector may be projected onto a byte scale for each of its 80 components. The byte 
scale ranges from 0 - 255 and preferably corresponds to 6 DRj lengths. All rotations and 
divisions are pre-computed into each vector SER(A). Using system 100 of the present 
invention, the transitive recognition space measure calculation MER(A,C) is carried out at a 
rate of 47 million per minute on the same computer system. This represents a many-fold 
improvement in recognition or matching speed. In addition, the template used for the initial 

20 

matching S(A) has a size 3.5 KBytes. The final template size of SER(A) is reduced to 80 Bytes. 
As will be appreciated, these dramatic improvements in speed and storage requirements are 
highly advantageous. 

While the invention has been described in conjunction with specific embodiments, it 
is evident that numerous alternatives, modifications, and variations will be apparent to those 

25 

skilled in the art in light of the foregoing description. 
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